SeNA-CNN: Overcoming Catastrophic Forgetting in Convolutional Neural Networks by Selective Network Augmentation

Lifelong learning aims to develop machine learning systems that can learn new tasks while preserving the performance on previous learned tasks. In this paper we present a method to overcome catastrophic forgetting on convolutional neural networks, that learns new tasks and preserves the performance on old tasks without accessing the data of the original model, by selective network augmentation. The experiment results showed that SeNA-CNN, in some scenarios, outperforms the state-of-art Learning without Forgetting algorithm. Results also showed that in some situations it is better to use SeNA-CNN instead of training a neural network using isolated learning.info:eu-repo/semantics/publishedVersio

Molecular and cellular limits to somatosensory specificity

Animals detect environmental changes through sensory neural mechanisms that enable them to differentiate the quality, intensity and temporal characteristics of stimuli. The 'doctrine of specific nervous energies' postulates that the different sensory modalities experienced by humans result of the activation of specific nervous pathways. Identification of functional classes of sensory receptors provided scientific support to the concept that somatosensory modalities (touch, pain, temperature, kinesthesis) are subserved by separate populations of sensory receptor neurons specialized in detecting innocuous and injurious stimuli of different quality (mechanical forces, temperature, chemical compounds). The identification of receptor proteins activated by different physicochemical stimuli, in particular ion channels of the Transient Receptor Potential (TRP) superfamily, has put forward the concept that specificity of peripheral sensory receptor neurons is determined by their expression of a particular "molecular sensor" that confers to each functional type its selectivity to respond with a discharge of nerve impulses to stimuli of a given quality. Nonetheless, recent experimental data suggest that the various molecular sensors proposed as specific transducer molecules for stimuli of different quality are not as neatly associated with the distinct functional types of sensory receptors as originally proposed. First, many ion channel molecules initially associated to the transduction of only one particular form of energy are also activated by stimuli of different quality, implying a limited degree of specificity in their transducing capacities. Second, molecular sensors associated with a stimulus quality and hence to a sensory receptor type and ultimately to a sensory modality may be concomitantly expressed in sensory receptor neurons functionally defined as specific for another stimulus quality. Finally, activation of voltage gated channels involved primarily in nerve impulse generation can also influence the gating of transducing channels, dramatically modifying their activation profile. Thus, we propose that the capacity exhibited by the different functional types of somatosensory receptor neurons to preferentially detect and encode specific stimuli into a discharge of nerve impulses, appears to result of a characteristic combinatorial expression of different ion channels in each neuronal type that finally determines their transduction and impulse firing properties. Transduction channels don't operate in isolation and their cellular context should also be taken into consideration to fully understand their function. Moreover, the inhomogeneous distribution of transduction and voltage-gated channels at soma, axonal branches and peripheral endings of primary sensory neurons influences the characteristics of the propagated impulse discharge that encodes the properties of the stimulus. Alteration of this concerted operation of ion channels in pathological conditions may underlie the changes in excitability accompanying peripheral sensory neuron injuries

Factors Affecting Frequency Discrimination of Vibrotactile Stimuli: Implications for Cortical Encoding

BACKGROUND: Measuring perceptual judgments about stimuli while manipulating their physical characteristics can uncover the neural algorithms underlying sensory processing. We carried out psychophysical experiments to examine how humans discriminate vibrotactile stimuli. METHODOLOGY/PRINCIPAL FINDINGS: Subjects compared the frequencies of two sinusoidal vibrations applied sequentially to one fingertip. Performance was reduced when (1) the root mean square velocity (or energy) of the vibrations was equated by adjusting their amplitudes, and (2) the vibrations were noisy (their temporal structure was irregular). These effects were super-additive when subjects compared noisy vibrations that had equal velocity, indicating that frequency judgments became more dependent on the vibrations' temporal structure when differential information about velocity was eliminated. To investigate which areas of the somatosensory system use information about velocity and temporal structure, we required subjects to compare vibrations applied sequentially to opposite hands. This paradigm exploits the fact that tactile input to neurons at early levels (e.g., the primary somatosensory cortex, SI) is largely confined to the contralateral side of the body, so these neurons are less able to contribute to vibration comparisons between hands. The subjects' performance was still sensitive to differences in vibration velocity, but became less sensitive to noise. CONCLUSIONS/SIGNIFICANCE: We conclude that vibration frequency is represented in different ways by different mechanisms distributed across multiple cortical regions. Which mechanisms support the “readout” of frequency varies according to the information present in the vibration. Overall, the present findings are consistent with a model in which information about vibration velocity is coded in regions beyond SI. While adaptive processes within SI also contribute to the representation of frequency, this adaptation is influenced by the temporal regularity of the vibration

Analysis of spatial relationships in three dimensions: tools for the study of nerve cell patterning

<p>Abstract</p> <p>Background</p> <p>Multiple technologies have been brought to bear on understanding the three-dimensional morphology of individual neurons and glia within the brain, but little progress has been made on understanding the rules controlling cellular patterning. We describe new matlab-based software tools, now available to the scientific community, permitting the calculation of spatial statistics associated with 3D point patterns. The analyses are largely derived from the Delaunay tessellation of the field, including the nearest neighbor and Voronoi domain analyses, and from the spatial autocorrelogram.</p> <p>Results</p> <p>Our tools enable the analysis of the spatial relationship between neurons within the central nervous system in 3D, and permit the modeling of these fields based on lattice-like simulations, and on simulations of minimal-distance spacing rules. Here we demonstrate the utility of our analysis methods to discriminate between two different simulated neuronal populations.</p> <p>Conclusion</p> <p>Together, these tools can be used to reveal the presence of nerve cell patterning and to model its foundation, in turn informing on the potential developmental mechanisms that govern its establishment. Furthermore, in conjunction with analyses of dendritic morphology, they can be used to determine the degree of dendritic coverage within a volume of tissue exhibited by mature nerve cells.</p

A Comprehensive Workflow for General-Purpose Neural Modeling with Highly Configurable Neuromorphic Hardware Systems

In this paper we present a methodological framework that meets novel requirements emerging from upcoming types of accelerated and highly configurable neuromorphic hardware systems. We describe in detail a device with 45 million programmable and dynamic synapses that is currently under development, and we sketch the conceptual challenges that arise from taking this platform into operation. More specifically, we aim at the establishment of this neuromorphic system as a flexible and neuroscientifically valuable modeling tool that can be used by non-hardware-experts. We consider various functional aspects to be crucial for this purpose, and we introduce a consistent workflow with detailed descriptions of all involved modules that implement the suggested steps: The integration of the hardware interface into the simulator-independent model description language PyNN; a fully automated translation between the PyNN domain and appropriate hardware configurations; an executable specification of the future neuromorphic system that can be seamlessly integrated into this biology-to-hardware mapping process as a test bench for all software layers and possible hardware design modifications; an evaluation scheme that deploys models from a dedicated benchmark library, compares the results generated by virtual or prototype hardware devices with reference software simulations and analyzes the differences. The integration of these components into one hardware-software workflow provides an ecosystem for ongoing preparative studies that support the hardware design process and represents the basis for the maturity of the model-to-hardware mapping software. The functionality and flexibility of the latter is proven with a variety of experimental results

Can computational efficiency alone drive the evolution of modularity in neural networks?

Some biologists have abandoned the idea that computational efficiency in processing multipart tasks or input sets alone drives the evolution of modularity in biological networks. A recent study confirmed that small modular (neural) networks are relatively computationally-inefficient but large modular networks are slightly more efficient than non-modular ones. The present study determines whether these efficiency advantages with network size can drive the evolution of modularity in networks whose connective architecture can evolve. The answer is no, but the reason why is interesting. All simulations (run in a wide variety of parameter states) involving gradualistic connective evolution end in non-modular local attractors. Thus while a high performance modular attractor exists, such regions cannot be reached by gradualistic evolution. Non-gradualistic evolutionary simulations in which multi-modularity is obtained through duplication of existing architecture appear viable. Fundamentally, this study indicates that computational efficiency alone does not drive the evolution of modularity, even in large biological networks, but it may still be a viable mechanism when networks evolve by non-gradualistic means

Symmetric Sensorimotor Somatotopy

BACKGROUND: Functional imaging has recently been used to investigate detailed somatosensory organization in human cortex. Such studies frequently assume that human cortical areas are only identifiable insofar as they resemble those measured invasively in monkeys. This is true despite the electrophysiological basis of the latter recordings, which are typically extracellular recordings of action potentials from a restricted sample of cells. METHODOLOGY/PRINCIPAL FINDINGS: Using high-resolution functional magnetic resonance imaging in human subjects, we found a widely distributed cortical response in both primary somatosensory and motor cortex upon pneumatic stimulation of the hairless surface of the thumb, index and ring fingers. Though not organized in a discrete somatotopic fashion, the population activity in response to thumb and index finger stimulation indicated a disproportionate response to fingertip stimulation, and one that was modulated by stimulation direction. Furthermore, the activation was structured with a line of symmetry through the central sulcus reflecting inputs both to primary somatosensory cortex and, precentrally, to primary motor cortex. CONCLUSIONS/SIGNIFICANCE: In considering functional activation that is not somatotopically or anatomically restricted as in monkey electrophysiology studies, our methodology reveals finger-related activation that is not organized in a simple somatotopic manner but is nevertheless as structured as it is widespread. Our findings suggest a striking functional mirroring in cortical areas conventionally ascribed either an input or an output somatotopic function

New insights into the classification and nomenclature of cortical GABAergic interneurons.

A systematic classification and accepted nomenclature of neuron types is much needed but is currently lacking. This article describes a possible taxonomical solution for classifying GABAergic interneurons of the cerebral cortex based on a novel, web-based interactive system that allows experts to classify neurons with pre-determined criteria. Using Bayesian analysis and clustering algorithms on the resulting data, we investigated the suitability of several anatomical terms and neuron names for cortical GABAergic interneurons. Moreover, we show that supervised classification models could automatically categorize interneurons in agreement with experts' assignments. These results demonstrate a practical and objective approach to the naming, characterization and classification of neurons based on community consensus

Beyond Statistical Significance: Implications of Network Structure on Neuronal Activity

It is a common and good practice in experimental sciences to assess the statistical significance of measured outcomes. For this, the probability of obtaining the actual results is estimated under the assumption of an appropriately chosen null-hypothesis. If this probability is smaller than some threshold, the results are deemed statistically significant and the researchers are content in having revealed, within their own experimental domain, a “surprising” anomaly, possibly indicative of a hitherto hidden fragment of the underlying “ground-truth”. What is often neglected, though, is the actual importance of these experimental outcomes for understanding the system under investigation. We illustrate this point by giving practical and intuitive examples from the field of systems neuroscience. Specifically, we use the notion of embeddedness to quantify the impact of a neuron's activity on its downstream neurons in the network. We show that the network response strongly depends on the embeddedness of stimulated neurons and that embeddedness is a key determinant of the importance of neuronal activity on local and downstream processing. We extrapolate these results to other fields in which networks are used as a theoretical framework

The effect of water immersion on short-latency somatosensory evoked potentials in human

<p>Abstract</p> <p>Background</p> <p>Water immersion therapy is used to treat a variety of cardiovascular, respiratory, and orthopedic conditions. It can also benefit some neurological patients, although little is known about the effects of water immersion on neural activity, including somatosensory processing. To this end, we examined the effect of water immersion on short-latency somatosensory evoked potentials (SEPs) elicited by median nerve stimuli. Short-latency SEP recordings were obtained for ten healthy male volunteers at rest in or out of water at 30°C. Recordings were obtained from nine scalp electrodes according to the 10-20 system. The right median nerve at the wrist was electrically stimulated with the stimulus duration of 0.2 ms at 3 Hz. The intensity of the stimulus was fixed at approximately three times the sensory threshold.</p> <p>Results</p> <p>Water immersion significantly reduced the amplitudes of the short-latency SEP components P25 and P45 measured from electrodes over the parietal region and the P45 measured by central region.</p> <p>Conclusions</p> <p>Water immersion reduced short-latency SEP components known to originate in several cortical areas. Attenuation of short-latency SEPs suggests that water immersion influences the cortical processing of somatosensory inputs. Modulation of cortical processing may contribute to the beneficial effects of aquatic therapy.</p> <p>Trial Registration</p> <p>UMIN-CTR (UMIN000006492)</p